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ABSTRACT 

A unified overview is given of problem formulation approaches for the optimization of 
multidisciplinary coupled systems. The overview includes six fundamental approaches upon 
which a large number of variations may be made. Consistent approach names and a compact 
approach notation are given. The approaches are formulated to apply to general nonhier- 
archic systems. The approaches are compared both from a computational viewpoint and 
a managerial viewpoint. Opportunities for parallelism of both computation and manpower 
resources are discussed. Recommendations regarding the need for future research are ad- 
vanced. 


‘This research was supported by the National Aeronautics and Space Administration under NASA Con- 
tract No. NAS1-19480 while the first author was in residence at the Institute for Computer Applications in 
Science and Engineering (ICASE), NASA Langley Research Center, Hampton, VA 23681=0001. 


i 



\ 1 ! 



I. Introduction 


The increased demands of economic competition and the complexity of engineering systems have led to 
the rapid growth of the field of multidisciplinary design optimization (MDO) over the past decade (AIAA 
White Paper 1991). The field of MDO is concerned with how to analyze efficiently and design optimally 
a system governed by multiple coupled disciplines or made up of coupled components. It is a part of 
the concurrent engineering technology which may well be an enabling technology for complex advanced 
systems (Sobieszczanski-Sobieski and Tulinius 1991). 

Several approaches to problem formulation have emerged in a rather ad hoc fashion over the years. 
A unified overview of MDO approaches is given in this paper. Fundamental approaches are defined and 
endowed with consistent names, and a compact notation is introduced which uniquely distinguishes 
between the large number of variations which exist. Some of the fundamental approaches presented here 
have been significantly modified from the way they were originally introduced, and at least one may be 
regarded as new. This overview was inspired by the previous work of (Cramer et al. 1993) which was 
one of the first attempts to classify approaches to MDO. That work was limited to single-level 
optimization approaches while the overview presented in this paper covers both single and multilevel 
optimization approaches. 

In the past, MDO approaches have been categorized as either hierarchic or nonhierarchic according 
to the types of systems to which they apply (Sobieszczanski-Sobieski 1990a). In hierarchic systems, 
children disciplines are coupled only to parent disciplines and not to each other. Nonhierarchic systems 
are more general since no restrictions are placed on how disciplines are coupled. MDO approaches are 
presented here in a form applicable to general nonhierarchic systems. Approaches which have traditionally 
been thought to be limited to hierarchic systems are formulated here to apply to nonhierarchic systems. 

The fundamental approaches are compared in this paper. Typical values for problem size and iteration 
parameters are assumed, and the order of magnitude of computational effort is estimated for several test 
cases and implementation options. Managerial strengths and weaknesses of the fundamental approaches 
are also noted. Finally, recommendations for needed research are advanced. 


II. A Coupled System 

Consider the three-discipline system shown in Figure 1 as a basis for discussion. Each box in the figure 
represents a module that transforms input to output. These modules may correspond to disciplines or 
components, but for uniformity the term "discipline" will be used throughout to refer to a module. The 
three-discipline system is small enough to keep the discussion simple but large enough to develop patterns 
that extend to systems that include more disciplines. The system might be an aircraft wing where 
Disciplines 1, 2, and 3 represent aircraft performance, aerodynamics, and structures, respectively. 
Alternatively, the disciplines might represent physical components of a system such as the fuselage, the 
wings, and the propulsion system. In either case, the disciplines are coupled to each other by the output- 
to-input data exchanges. The variables and functions shown in Figure 1 will now be explained. Table 1 
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identifies what these variables and functions represent for the aircraft wing example. The reader is 
encouraged to refer back to this table for examples of the abstract definitions which follow. 

It is assumed that all functions are calculated within the disciplines. This calculation operates on a 
set of state equations for values of state variables . The vectors s„ s 2 , and s 3 contain the state variables for 
each of the disciplines. 

The vectors r„ r 2 , and r 3 are the residuals in the state equations for the disciplines. Disciplinary 
analyzers seek values for the state variables which reduce the residuals in the state equations to zero. 
Disciplinary evaluators evaluate the residuals in the state equations for given values of the state variables. 
Obviously, the computational effort required by a disciplinary evaluator is significantly less than that 
required by a disciplinary analyzer. An additional task that is embedded in either disciplinary analyzers 
or disciplinary evaluators is computation of the coupling functions, the design constraint functions, and 
the design objective functions as it will be explained later. 

The vectors y 12 , y 13 , y 21 , y^, y 31 , and y 32 are the coupling functions. Note that y y contains those 
functions computed in Discipline i which are needed as input to Discipline j. It is these coupling functions 
which complicate the order of execution of the disciplinary analyzers (or evaluators). By associating with 
each vector of coupling functions a corresponding vector of coupling variables (y 12 *, y 13 *, y 2] *, y^*, y 3) *, 
and y 32 *), the disciplinary analyzers (or evaluators) may be executed in parallel. Each receives coupling 
variables as input and computes coupling functions as output. One of the tasks of MDO is to satisfy 
coupling equality between each coupling variable and its corresponding coupling function. 

The vectors x, x w x 2 , and x 3 are mutually disjoint sets of design variables. Design variables 
represent the independent inputs which distinguish one design from another. Note that x contains system 
design variables needed by more than one discipline. The vectors x 1? x 2 , and x 3 contain disciplinary design 
variables associated with Disciplines 1, 2, and 3, respectively. 

The vectors g„ g 2 , and g 3 contain the design constraint functions. The design constraints normally 
guard against failure and otherwise unacceptable behavior. Only inequality constraints are considered here, 
and it is assumed that each constraint has been formulated such that zero is its allowable value, and it is 
satisfied when less than zero. 

The vectors fj, f^ and f 3 contain the design objective functions. As mentioned before, the design 
constraint and design objective functions are computed either in disciplinary analyzers or disciplinary 
evaluators. Design objectives normally minimize cost and ma ximiz e benefits, and therefore, may be 
competing. It is assumed that each objective has been formulated such that it is improved through 
minimization, and the value of zero is associated with its selected target value. It is possible to cast 
objectives as constraints and vice versa. Thus, constraint and objective functions will collectively be 
called design functions, and their associated inequality constraints and minimization will collectively be 
called the design criteria . 

The task at hand is to determine the values of the design, state, and coupling variables which satisfy 
the state equations, the coupling equalities, and the design criteria. 
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III. The Fundamental Approaches 


The fundamental approaches to MDO formulation vary in two aspects. First is the distinction between 
single-level optimization and multilevel optimization . In multilevel optimization approaches, the 
disciplinary design variables are determined by disciplinary optimizers and the system design variables 
are determined by the system optimizer . In single-level optimization approaches, both disciplinary and 
system design variables are determined by the system optimizer. 

Second is the choice between simultaneous analysis and design (SAND) and nested analysis and 
design (NAND). This distinction can be made at both the system and the discipline levels. At the 
discipline level, SAND implies that the disciplinary design and state variables are determined 
simultaneously by the optimizer, while NAND implies that the optimizer determines only the disciplinary 
design variables and requires determination of the state variables at each iteration. Thus, at each iteration 
of the optimizer, disciplinary evaluators are called for SAND while disciplinary analyzers are called for 
NAND. 

At the system level, SAND implies that system design variables and coupling variables are determined 
simultaneously by the system optimizer, while NAND implies that the system optimizer determines only 
the system design variables and requires determination of the coupling variables at each iteration by calling 
a system analyzer . 

Each fundamental approach has a three part name. The first part indicates whether the approach is 
a multilevel optimization or a single-level optimization approach. The middle and last parts of the name 
indicate SAND vs. NAND at the system and discipline levels, respectively. 


Single-SAND-SAND 

In this approach all variables are determined simultaneously by the system optimizer, and disciplinary 
evaluators are called at each iteration to compute all the functions. Figure 2 displays boxes corresponding 
to the system optimizer and to the disciplinary evaluators. This figure, as well as those which follow, 
shows where the data originate and where they are being used. To that end, individual variables and 
functions are listed within the boxes where they are determined, and the arrows point to the boxes where 
each is needed. The figures are not flowcharts and do not convey information about sequencing of the 
execution of the boxes, except that the execution of the boxes to the right is nested within the execution 
of the boxes to the left. The execution sequencing is explained in the narrative and by using a special 
notation introduced later. The Single-SAND-SAND approach has been referred to as the "All-At-Once" 
approach (Cramer et al. 1993). It has been demonstrated on single-discipline examples (Haftka 1985, 
Ta’asan et al. 1992, Hou et al. 1993), but multidisciplinary applications are scarce. The system 
optimization problem is: 

Find: f,x,x 1 ,x 2 ,x 3 ,y 12 *,y 1 3 *,y 21 *,y 2 3 + ,y 3 ] *,y 3 2 *,s 1 ,s 2 ,s 3 
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Minimize: 

f 



Satisfy: 

gi < o> 

g 2 < o, 

g 3 <o. 


f ! < f. 

f 2 <f, 

f 3 <f, 


yi2 = yi2*> 

* ~ 
m 

>-> 

II 

£ 



y 2J = y2i*> 

y23 = y23*> 



y 3 i = y 3J *> 

y 32 = y 32 *> 



r, = 0, 

II 

© 

r 3 = 0 


Note the addition of the scalar design variable, f, the system objective. By minimizing this variable 
and constraining it to be greater than each of the disciplinary objectives, the maximum of the objectives 
is effectively minimized. This multiobjective formulation is known as the "minimax formulation," and 
is just one of several formulations. Note that the minimax formulation treats all design functions as 
inequalities whether they be constraints or objectives. As with all multiobjective formulations, the results 
largely depend on the scaling and the choice of allowable/target values of the design functions. 


Single-SAND-NAND 

This approach is like the Single-SAND-SAND approach except that disciplinary analyzers rather than 
disciplinary evaluators are called at each iteration of the system optimizer as shown in Figure 3. Note the 
disappearance of the state variables and the residuals since these become internal to the disciplinary 
analyzers. This approach is also referred to as the "Individual Discipline Feasible" approach (Cramer et al. 
1993). It has been demonstrated on nonhierarchic, multidisciplinary examples (Haftka et al. 1992). The 
system optimization problem is: 


Find: f,x,x 1 ,x 2 ,x 3 ,y 12 ^ 13 ^ 2 i*,y 23 *,y 31 *,y 32 * 

Minimize: f 


Satisfy: 


g, <0, g 2 < 0, g 3 < 0, 

f i < f, f 2 < f» f 3 < f, 

yn = yt2*> yis = yo*. 

y 2 i = y 2 i*> y23 = yzA 

y 3 i = y 31 *. y 32 = y 32 * 
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Single-NAND-NAND 


In this approach, a system analyzer is called at each iteration of the system optimizer to determine the 
values of the coupling variables which match their corresponding coupling functions. The system analyzer 
itself is typically iterative, and it calls upon disciplinary analyzers at each iteration as shown in Figure 4. 
This approach is also referred to as the "Multidiscipline Feasible" approach (Cramer et al. 1993). It has 
been demonstrated more than any other approach on nonhierarchic, multidisciplinary examples (Grossman 
et al. 1989, Hajela et al. 1990, Sobieszczanski-Sobieski et al. 1991, Haftka et al. 1992). The system 
optimization problem is: 

Find: f,x,x!,x 2 ,x 3 

Minimize: f 

Satisfy: g 1 <0, g 2 < 0, g 3 < 0, 

f, < f, f 2 < f, f 3 < f 

The first three fundamental approaches which have now been presented illustrate the difference 
between SAND and NAND. The NAND approaches have received the most attention. The SAND 
approaches add coupling and state variables to the set of variables determined by the optimizer. It is 
likely that this increased set of variables will affect the convergence properties of the entire process. 
Whether the total computational effort increases or decreases depends on the problem at hand. 


Multi-SAND-SAND 

In this approach, disciplinary optimizers which simultaneously determine the disciplinary design and state 
variables are called at each iteration of the system optimizer as shown in Figure 5. The system optimizer 
simultaneously determines the system design and coupling variables. This new approach has never been 
demonstrated. The system optimization problem is: 

Find: f J x,y 12 *,y 13 *,y 21 *,y 23 *,y 31 *,y 32 * 

Minimize: f 

Satisfy: dj <0, d 2 < 0, d 3 < 0 

The scalars d } , d 2 , and d 3 are discrepancy functions computed by the disciplinary optimizers. The 
disciplinary optimization problem for Discipline i is: 
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Find: 

x i> s i>dj 



Minimize: 

di 



Satisfy: 

& < dj* 

iff < d i; 

(1) 


yij-y,j* < di, 

yij*-y y < d s , for j*i 

(2) 


h < dj, 

-*i < d i 

(3) 


Note that the disciplinary optimizer seeks to minimize the maximum discrepancy in the design criteria (1), 
the coupling equality (2), and the state equations (3). This max norm formulation may cause the 
discrepancy functions in the system optimization problem to be nonsmooth, and an appropriate system 
optimization algorithm should be used (Balling and Sobieszczanski-Sobieski 1994). 


Multi-SAND-NAND 

The only difference between this approach and the Multi-SAND-SAND approach is that disciplinary 
analyzers are called at each iteration of the disciplinary optimizers as shown in Figure 6. The system 
optimization problem is identical to the system optimization problem for the Multi-SAND-SAND 
approach. The disciplinary optimization problem for Discipline i is: 

Find: Xj,dj 

Minimize: d ; 

Satisfy: & < d j5 f r f < d,, ( 4 ) 

Yij-y.j* < dj, y^-yy < d, for j*i ( 5 ) 

This approach has been referred to as "multilevel optimization by hierarchic linear decomposition" 
(Sobieszczanski-Sobieski 1982). Various forms of the approach have been widely demonstrated on 
hierarchic, multidisciplinary examples (Wrenn and Dovi 1987; Walsh et al. 1994). Note that the 
formulation given above is equally applicable to nonhierarchic examples. Nevertheless, it has not yet been 
demonstrated on nonhierarchic multidisciplinary examples. 

Again, note the minimization of the max norm of the discrepancy in design criteria (4) and coupling 
equality (5). Alternative formulations for the disciplinary optimization problem have been suggested. 
Originally it was proposed to minimize the norm of discrepancy in design criteria while satisfying 
coupling equality (Sobieszczanski-Sobieski 1982): 

Find: Xj,d, 
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Minimize: 


di 

Satisfy: g, < d„ f r f < d ( , 

Yij = Yij* for j*i 

The converse formulation which minimizes the norm of discrepancy in coupling equality while 
satisfying the design criteria has also been suggested (Schmit and Ramanathan 1973; Sobieszczanski- 
Sobieski 1993): 

Find: Xj,dj 

Minimize: d, 

Satisfy: g, <0, f ( < f, 

yij-yu* <d i> y«*-yij< d i for j^i 

Both of these latter formulations suffer from the possibility that there may be no feasible solution to 
disciplinary optimization problem given the values of the system design variables and coupling variables 
sent down from the system. 

Alternatives to the max norm have also been proposed for the disciplinary optimization problem. One 
family of norms that has been suggested is the "KS family" named for (Kreisselmeier and Steinhauser 
1983): 

Find: Xj 

Minimize: d, = (1/p) ln{ 2exp{pg i ) + 2exp(p(f i -f)) + (l/2)X i . i [2exp(p(y ir y ij + )) + Sexp(p(y jj *-y ij ))] } 

The parameter p is a positive real number, the sum 2j»i is over the disciplines other than Discipline i, 
and the other sums are over the elements of the vectors involved. As p goes to infinity, the KS norm 
becomes equivalent to the max norm. Otherwise, the KS norm is greater than the max norm by an 
amount which is bounded by (l/p)ln(m) where m is the sum of the lengths of the vectors gj, f j( and y^ for 

H- 

Another family of norms that was used in (Schmit and Ramanathan 1973) is the "L family": 

Find: Xj 

Minimize: d ( = { S(max(0,g,)) p + 2(max(0,f r f)) p + 2 j- j[Z|y ij -y s *| p ] } ,/p 

The parameter p is a positive integer and the sums are the same as for the KS family of norms. The 
1 2 norm is the familiar Euclidean norm, and as p goes to infinity, the lp norm becomes equivalent to the 
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max norm. Further discussion of the use of these norms in MDO is given elsewhere (Balling and 
Sobieszczanski-Sobieski 1994). 


Multi-NAND-NAND 

It is difficult to formulate an approach which utilizes both disciplinary optimizers and a system analyzer. 
An adequate formulation which nests the former inside the latter has not been developed. A Multi- 
NAND-NAND formulation which first executes the system analyzer and then executes the disciplinary 
optimizers is possible as shown in Figure 7. Thus, at each iteration of the system optimizer, the 
disciplinary design variables are held fixed while a system analyzer is executed to determine the coupling 
variables, and then the coupling variables are held fixed while disciplinary optimizers are executed to 
determine the disciplinary design variables. Disciplinary analyzers are called at each iteration of both the 
system analyzer and the disciplinary optimizers. 

A Multi-NAND-NAND approach was first proposed in (Sobieszczanski-Sobieski 1988) and 
subsequently developed, augmented with heuristics, and demonstrated on nonhierarchic, multidisciplinary 
examples (Bloebaum 1991; Eason et al. 1994). The formulation presented here is somewhat similar. In 
addition to determining the coupling variables, the system analyzer computes a scalar cumulative design 
function for each discipline: q = max(g i ,f i -f). These functions as well as their sensitivities with respect 
to disciplinary design variables are sent as constants to the disciplinary optimizers. The optimization 
problem for Discipline i is: 

Find: Xj,dj 

Minimize: d; 

Satisfy: gr c iOp„) < <Mi*f-Cj(l-Pa) < d,, ( 6 ) 

c^-CjU-Pu) < di for M ( 7 ) 

The functions q* for j^i are first-order approximations of the cumulative design functions for the other 
disciplines based on the constants and sensitivities received from the system analyzer. Thus, Discipline i 
has some responsibility for satisfying the design constraints of the other disciplines. The scalars p fj for 
j*i in (7) are the responsibility fractions allocated to Discipline i. Similarly, some of the responsibility 
for satisfying the design constraints of Discipline i is shifted to the other disciplines. The scalar p u in (6) 
is the responsibility fraction that remains with Discipline i. The system optimization problem insures that 
the responsibility fractions sum to unity: 

Find: f> X >Pll>Pl2>Pl3>P21’P22>P23>P31>P32>P33 

Minimize: f 
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Satisfy: 


dj <0, d 2 < 0, d 3 < 0, 

Pll + P21 + P31 = 1> Pl2 + P22 + P32 = Pl3 + P23 + P33 = 1 


IV. Variations on the Fundamental Approaches 

The fundamental approaches may be modified by: 1) mixing, 2) sequencing, and 3) composing. The 
fundamental approaches may be mixed by varying the usage of disciplinary optimizers, analyzers, and 
evaluators across disciplines. For example, one discipline may possess several disciplinary design 
variables while another may have few or even none. It may be wise to employ a disciplinary optimizer 
with the former (multilevel approach) but not with the latter (single-level approach). Similarly, 
disciplinary analyzers may be used in some disciplines (NAND approach) while disciplinary evaluators 
are used in others (SAND approach). 

Sequencing requires that disciplinary analyzers (or evaluators) are executed sequentially rather than 
in parallel. This allows deletion of some of the coupling variables. In the three-discipline example, if 
Analyzer 1 is always executed before Analyzers 2 and 3 which are still executed in parallel, the coupling 
functions y 12 and y 13 output from Analyzer 1 may be sent directly as input to Analyzers 2 and 3, and there 
is no need for the coupling variables y, 2 * and y 13 *. However, the coupling variables y 21 *, y B *, y 3I *, and 
y 32 * are still needed. The loss of parallelism may be small if Analyzer 1 is significantly longer 
computationally than Analyzers 2 and 3. 

Composing allows one to treat a disciplinary optimizer in a multilevel approach as a system optimizer 
for a system within the discipline. In this way multilevel approaches may be extended to more than two 
levels. For example, because of the potentially large number of structural design variables, the structures 
discipline may be treated as a coupled system comprised of subdisdplines corresponding to spars, ribs, 
stringers, and skin as well as the assembled structure. Optimizers, analyzers, and/or evaluators may then 
be employed for each subdiscipline. The coupling variables between the subdisciplines in the structural 
system are viewed as the state variables of the structures discipline in the wing system. 

Because of the large number of possible variations, it is impractical to try to give names to each. 
Nevertheless, a compact notation which captures the subtle differences between variations can be 
introduced. In this notation: 


SO = 
SA = 
Ej = 

A = 
o i = 

[] = 


-> 


System Optimizer 
System Analyzer 
Evaluator Discipline i 
Analyzer Discipline i 
Optimizer Discipline i 
nested execution 
parallel execution 
sequential execution 
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Nested execution has the highest precedence followed by parallel execution and finally by sequential 
execution. Using this notation, the compact formulas for the fundamental approaches are: 

Single-SAND-SAND SO[E, || E 2 || E 3 ] 

Single-SAND-NAND SO[A, || A 2 1| A 3 ] 

Single-NAND-NAND SO[SA[A, || A 2 1| A 3 ]] 

Multi-SAND-SAND SOfO^J || 0 2 [EJ || 0 3 [E 3 ]] 

Multi-SAND-NAND S0[0,[A,] || 0 2 [AJ || 0 3 [A 3 ]] 


Multi-NAND-NAND SO[SA[A, || A 2 1| A 3 ] -> 0,[A,] || Q 2 [A 2 ] || 0 3 [A 3 ]] 


Each of the above formulas codes full information about the execution sequencing and may be 
regarded as a surrogate for a flowchart. Thus, a concise definition of each fundamental approach consists 
of its data flow diagram given in one of the Figures 2 to 7 and of its execution sequence recipe coded in 
its corresponding formula above. 

The following three-level approach is an interesting and plausible example which involves mixing, 
sequencing, and composing. Discipline 1 is aircraft performance. Discipline 2 is aerodynamics. Discipline 
3 is structures, Discipline 4 is the assembled structure, and Disciplines 5 through N are spars, ribs, 
stringers, skin, etc.: 


SO[Aj -> A 2 -> S0 3 [A 4 -> 0 5 [AJ || ... || O n [A n ]]] 


Discipline 3 is itself a system, and as with all systems, it is assumed that all design and coupling 
functions are computed within its disciplines (4 through N). Thus, there are no f’s, g’s, nor y’s with "3" 
subscripts. There may, however, be system design variables associated with the structures system, x 3 . 
Assume for now that Disciplines 5 through N are coupled only to Discipline 4, and not to each other nor 
to Disciplines 1 and 2. The optimization problem for the entire system is: 


Find: 

f,x,x„x 2 ,y 21 *,y 4 i*,y 42 * 

Minimize: 

f 

Satisfy: 

gi < 0, f r f < 0, 


g 2 < 0, f 2 -f < 0, 


d 3 < 0 
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The optimization problem for the structures system (System or Discipline 3) is: 


Find: x 3 ,x 4 ,y 54 *,...,y N4 *,d 3 

Minimize: d 3 


Satisfy: 


§4 < d 3 . 

y4i-y 4 i* < d 3 , 

y 4 2-y 4 2* < d 3 , 

d 4 < d 3 , 


f 4 -f < d 3 , 
y 4 i**y 4 i < d 3 , 
y 42 *-y 4 2 < d 3 , 
d N < d 3 


The optimization problem for Discipline i where i ranges from 5 to N is: 


Find: 

Minimize: 

Satisfy: 


M, 

4 

& <dj, 

y i4 -y i4 * < dj, 


f r f<d j , 

y»*-y« < dj 


V. Implementation Options 

A limited but necessary discussion of implementation options is given in this section. Different methods 
which may be implemented within evaluators, analyzers, and optimizers will be mentioned here to 
establish a background for the next section where computational effort is estimated. Since analyzers and 
optimizers may be gradient-based, methods of calculating sensitivities are also described. It should be 
mentioned that implementation options also include the choice between serial and parallel computing 
which will be treated in the next section rather than here. 


Disciplinary Analyzers and Evaluators 

Disciplinary evaluators determine the residuals in the state equations while disciplinary analyzers determine 
the values of the state variables which reduce the residuals to zero. Most existing engineering software 
packages are of the disciplinary analyzer variety. They typically exploit specialized techniques and models 
which have evolved within the discipline, and they may solve the state equations directly or iteratively. 

The derivatives of outputs from disciplinary analyzers or evaluators with respect to inputs are referred 
to as disciplinary sensitivities . These sensitivities may always be computed by a finite difference 
approximation (FD), however, computational efficiency and accuracy may be gained by directanalytical 
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sensitivity (DAS) based on implicit differentiation of the state equations (Adelman and Haftka 1986). This 
option has become available in an increasing number of analyzers. A recently introduced alternative is 
"automatic differentiation" (Bischof et al. 1992). 


Disciplinary Optimizers 

Disciplinary optimizers solve the disciplinary optimization problems of the various multilevel optimization 
approaches. These optimizers call disciplinary analyzers or evaluators to compute their constraints. In 
some disciplines, specialized techniques and models have been developed for efficient optimization (e.g., 
the optimality criteria methods of the structures discipline). In other disciplines, general-purpose nonlinear 
programming algorithms may be used. Some disciplines might involve combinatorial or stochastic search 
if discrete or topological variables are present. Still in other disciplines, optimization may be quite 
informal consisting of empirical rules and procedures of design practice. 

The only function that a disciplinary optimizer returns to the system optimizer is its objective, the 
discrepancy function dj Calculation of the derivative of the optimal objective with respect to fixed 
parameters is known as optimum sensitivity analysis (OSA). It has been shown that when formal 
optimization algorithms are used in disciplinary optimization, OSA can be calculated directly from the 
Lagrange multipliers and disciplinary sensitivities (Barthelemy and Sobieszczanski-Sobieski 1983). 


System Analyzer 

The system analyzer determines the values of coupling variables which match their corresponding coupling 
functions as computed by disciplinary analyzers. There are two basic iterative techniques commonly used. 
The first is fixed-point iteration in which the coupling functions computed from one iteration are used as 
the coupling variables at the start of the next iteration. Iterations continue until the change in coupling 
variables is negligible. The second technique is Newton’s method which employs not only the coupling 
functions but their derivatives with respect to the coupling variables at each iteration in order to accelerate 
convergence. Sequencing may be applied to either technique to eliminate some of the coupling variables 
altogether. 

The derivatives of the converged coupling variables can be computed by solving a linear set of 
equations known as the global sensitivity equations (GSE) (Sobieszczanski-Sobieski 1990b). The 
coefficients and right-hand-side vectors in the GSE consist of disciplinary sensitivities. The GSE method 
for system sensitivity analysis is an alternative to FD at the system level, just as DAS is an alternative to 
FD at the disciplinary level. 
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System Optimizer 


The system optimizer solves the system optimization problem of the various approaches. This optimizer 
may call disciplinary optimizers, analyzers, and/or evaluators as well as a system analyzer. Therefore, it 
is imperative that the system optimizer calls for constraint evaluation as few times as possible. It is 
recommended that the system optimizer repeatedly constructs approximations of the constraints and 
optimizes the approximate problem. As this succession of approximate optimization problems proceeds, 
move limits on the variables should be adaptable to insure robust convergence. 

Two families of constraint approximations are possible. The first is the local approximation family 
in which the approximation is based solely on the exact values of the constraints and their gradients 
evaluated at the current design. A first-order approximation of the constraints can then be constructed. 
The second family is the global approximation family in which a few exact constraint evaluations are 
performed throughout optimization variable space, and the results are fitted with simple approximate 
functions (Box and Draper 1987). Statistical methods from the design of experiments are useful for 
selecting the best locations to perform exact constraint evaluation. 


VI. Computational Cost 

This section presents an attempt at comparing the fundamental approaches introduced in the foregoing, 
based on their computational cost regarded as the primary concern to the user. However, not all user’s 
concerns may be reduced to cost because, if one interprets the "disciplines" as human organizations, the 
managerial advantages to be discussed later become another set of considerations important to users. 


Computational Cost Measure 

Computational cost may be measured in a number of ways, e.g., the CPU time, the elapsed real time (ET), 
the required memory, etc. This section presents cost evaluation of the fundamental approaches introduced 
in the foregoing, based only on their ET for computing, exclusive of the time needed for the data 
transmission and other overhead functions. It is difficult to provide a truly objective cost evaluation 
because the problem dependence of the assumptions and parameters involved taints any cost comparison 
of the foregoing approaches as being speculative. Furthermore, there are many variations of the 
fundamental approaches as well as many options for serial implementation (SI) and parallel 
implementation (PI). 

To circumvent the above difficulties the system at hand was examined at only two levels of 
complexity as measured by the number of interacting disciplines: a very simple system of two disciplines, 
and one whose number of disciplines was increased by one order of magnitude (OOM) to twenty. Four 
implementation options were investigated as given in Table 2. 
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For each of the implementation options, the ET was calculated at each of the two levels of complexity 
using formulas that link it to the parameters of the optimization and analysis problems. The parameters 
that define the analysis and optimization dimensionalities and affect ET are the lengths NX, NX;, NY^, 
and NSj of the previously defined variable vectors x, x t , y^, and Sj. Additional parameters relative to the 
number of iterations required for the various iterative procedures will be defined later. These parameters 
were systematically varied in the ET calculations in a series of numerical experiments to obtain data for 
the cost comparison. 


Evaluation of ET 

The computation time estimates for a conventional serial implementation (SI) were based on the 
assumption that the ET value is proportional to the number of the floating-point multiplications (FLMs). 
Omitting the proportionality coefficient, the ET is simply equated to the FLM count formulas commonly 
used in numerical methods. 

The number of FLMs required to solve a nonsymmetric dense linear system of state equations for 
Discipline i by a direct method is approximately NSj 3 /3. If the state equations are nonlinear, then the 
direct method may be embedded within an iterative scheme, and the number of FLMs required for analysis 
may be approximated as: 


CA, = MA • NS| 3 /3 

where MA is the number of iterations. The above formula was used in the estimates of the ET given in 
this paper, even though it is realized that if the state equations are sparse, the CAj may be reduced 
significantly to CAj = b ■ MA * NSj 2 where coefficient b depends on the degree of sparsity. Furthermore, 
if the state equations are sparse and solved iteratively rather than directly, CA, may be reduced further to 
CAj = b • MI • NSj where MI is the number of iterations. 

The number of FLMs required to evaluate the state equations (rather than solve them) was taken as 
the product of a matrix and a vector, hence: 


CE, = NS 2 

The number of FLMs required to solve a dense linear optimization problem with NV optimization 
variables is approximately 1.5 • NV 3 . The optimization variables in the various system and disciplinary 
optimization problems depend on the approach and may include design, coupling, and state variables. If 
nonlinear optimization problems are solved by solving a sequence of linearized problems, the number of 
FLMs may be approximated as: 


CO = MO • 1.5 • NV 3 
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where MO is the number of linearized problems solved. The above formula was used in the estimates 
of the ET given in this paper. In addition to the above formula, each linearized problem requires a single 
sensitivity analysis (or evaluation). 

The number of FLMs required for a single sensitivity analysis with repect to NV variables depends 
on the implementation option: 

SI/FD: CS = 2i(CA, • (1+NV)) 

SI/DAS: CS = ^(CAi+CEi • NV) 

PI/FD: CS = maXj(CAj) 

PI/DAS: CS = maXj(CAj+CEj) 

where the sums and max’s are taken over the disciplines. Note that for the DAS implementations it was 
assumed that sensitivities can be calculated cheaply via backsubstitutions after the nonlinear iterations are 
converged. It should be noted that in the DAS implementations, sensitivities with respect to any state 
variables are cost-free since these sensitivities are simply the coefficients in the state equations themselves. 
Note that the parallel implementations assume an unlimited number of processors where each has enough 
memory to execute a disciplinary analysis. This is an assumption of a coarse-grained implementation on 
an "ultimate" machine which may either be a single multiprocessor computer or an equivalent computer 
network. It skirts the complex issues of the efficiency of parallel processing involved in the internal 
parallelization of an analysis code. 

In the estimates given in this paper, it was assumed that for those approaches involving a system 
analyzer, the system analyzer was based on fixed-point iteration. The parameter which indicates the 
number of required fixed-point iterations is MC. 

The number of FLMs required to triangularize the linear GSE equations is NY 3 /3 where NY is the 
total number of coupling variables for all disciplines. The number of FLMs required to solve the GSE 
equations by backsubstitution with respect to NV optimization variables is NV * NY 2 . 

The formulas used in this paper for the ET in terms of number of FLMs for the different approaches 
and implementation options are given in the Appendix. 


Numerical Experiments 

Experiments were conducted using the Appendix formulas to determine ET values and their dependence 
on selected parameters for the options from Table 2 and for tests cases described in this section. 

For comparison and display purposes, the ET data are normalized by the ET of the most expensive 
analysis. The base 10 logarithm is then taken of the normalized ET to emphasize its OOM rather than 
the ET value itself. The differences in ET are considered meaningful only if they exceed at least an half 
of an OOM. This is done to render the comparisons meaningful despite that it is a function of parameters 
that are very problem-dependent and are, therefore, "soft" data subject to assumptions of the parameter 
values that may vary over many OOM’s. 
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Two-Discipline Case 


For the two-discipline system, the baseline values of the problem parameters are suggested by the case 
of minimum weight optimization for a flexible wing treated as a system of two disciplines: aerodynamics 
and structures. The aerodynamic analysis (i=l), by a CFD code might be expected to be much more 
expensive in terms of ET than the structural finite element analysis (i=2), hence the setting of 
NS,=100,000 and NS 2 =10,000. The state variables for aerodynamics, s„ are the flow data at the grid 
points. For the structure they are the nodal displacements. The number of design variables x that affect 
directly both disciplines is set at NX = 10 to represent the overall wing shape geometry. The number of 
design variables x, that affect directly the aerodynamics only is NX,=10 to control the shape of those parts 
of the wing that are structurally inert (outside of the structural box), for instance, the leading edge radius. 
The variables x 2 are cross-sectional gages that affect directly the structures only. Typically, their number 
is larger than NX and NX,, hence, NX 2 =100. The coupling variables y 12 and y 21 are aerodynamic loads 
and structural displacements, respectively. Using a modal representation of both, their number may be 
kept reasonably low, e.g., NY 12 =NY 21 =50. 

Based on the experience (Sobieszczanski et al. 1976) one assumes MC=5. This low number is 
reasonable only if the wing structure had been initialized to be sufficiently stiff, otherwise MC might be 
very large or the iteration may not even converge. The implication is that the discipline of structures 
would do a certain amount of prerequisite design work, assuming a constant magnitude and distribution 
of aerodynamic loads, before the coupled problem optimization begins. Similarly it is known from 
experience (Schmit 1981) that MO may range for a low value of less than 10 to 40 and it is, therefore, 
set at MO=30. An assumed, nominal value for MA=5 was used. 


Twenty-Discipline Case 

A hypothetical system of twenty disciplines was assumed as an abstract example, not rooted in any 
specific application but intended to shed light on the use of the fundamental approaches in a large 
engineering system. The parameter values were taken as: NX=10, NX,=100, and NS—10,000 for all 
disciplines i, and NY S =5 for all discipline i-j pairs. These data defined Variant 1 of this case that will be 
referred to as "analysis-intensive" because the number of the state variables is so much larger than the 
number of the design variables. In Variant 2 of the same case, referred to as "design-intensive," the 
setting of NX,=NS;=1,000 was used. 


Experimental Results 

The ET data are displayed by means of line graphs in Figures 8, 9, and 10 which are constructed as 
follows. The vertical scale is in loglO of the normalized ET to illustrate the ET OOM as a multiple of 
that disciplinary analysis which is the most expensive one in the system at hand. To facilitate 
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comparisons, the vertical scale is the same in all three figures. Each of the fundamental approaches is 
represented in the graph by one line, labeled with an approach acronym consisting of the first letters from 
its first, middle, and last names. For comparison convenience, the line for each approach links the discrete 
data points corresponding to the implementation options from Table 2. Note that Options 3 and 4 from 
Table 2 are merged in the figures because it turned out that PI nearly obliterated the difference in ET 
between FD and DAS in all the tests. 

Scanning the graphs by eye conveys a lot of information. For example, the left-to-right scan of a 
particular line shows how sensitive the corresponding fundamental approach is to the implementation 
options, including the choice between the serial (SI) and parallel (PI) computing and the choice between 
finite differencing (FD) and analytical sensitivity analysis (DAS). The vertical scan reveals the differences 
between the fundamental approaches, accentuated by the use of solid and empty symbols for single-level 
and multilevel optimization approaches, respectively. Finally, simultaneous scanning of the graphs in 
Figures 8, 9, and 10 gives an idea about the trends generated by changing the dimensionality and nature 
of the problem. 

One must be cautious in drawing conclusions from the data in Figures 8, 9, and 10 about the relative 
cost advantages for each fundamental approach. For example, the ET reductions for FD from switching 
to PI from SI are shown by comparing the right-most and the left-most data point columns in these 
figures, and the corresponding benefit for DAS is illustrated by comparing the right-most and the middle 
data point columns in these figures. These benefits must be tempered by the realization that they are, 
partially at least, due to the unlimited machine assumption. In comparing the ET cost of FD and DAS, 
one should also remember the accuracy advantages of DAS over FD. Finally, one needs to bear in mind 
that the PI considered herein is coarse-grained, hence not representative of the full potential of parallel 
computing. This assumption is, at least to a degree, the cause of no ET reduction due to PI in S-S-S. 
Finally, a reminder is in order that ET differences below 0.5 OOM are not significant in the comparisons. 

The assumed values for the iteration parameters MA, MC, and MO were increased, one at a time, by 
a factor of 10 to assess the effect on the relative ranking of the approaches. Graphs similar to Figures 8, 
9, and 10 were produced. Increasing MA tended to reduce ET in the SAND-SAND approaches relative 
to the others. Increasing MC increased ET in the SAND-NAND approaches, while increasing MO 
increased ET in the multilevel approaches. The FLM count formulas for analysis and optimization were 
also reduced to account for sparsity. This tended to reduce the SAND-SAND ET relative to the others. 


VII. Managerial Considerations in the Choice of Approach 

These considerations may be as important as the ET comparisons. To develop a broad work front of 
people and machines, an engineering organization is customarily divided into groups that correlate with 
the disciplines in the fundamental approaches discussed herein. These groups are intended to work 
concurrently. The resulting compression of the project calendar time is the same motivation that underlies 
the development of concurrent processing in computer technology. 
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Clearly, the need to enable the human organization groups to work concurrently is supported by the 
multilevel optimization approaches. These approaches have additional advantage because they allow 
disciplinary specialists to use their own methods, including informal ones, that have evolved over time, 
to solve their analysis and optimization problems autonomously. This includes the determination within 
disciplines of the values of disciplinary design variables and functions and may even extend to choices 
among discrete design alternatives. This is beneficial not only to the pace and quality of work but also 
to the human motivation. 

Additional consideration is the dependence, at least in short-to-mid term, on the existing analysis and 
optimization software. The same approaches that favor autonomy of the disciplines preserve also usability 
of that software. 

Finally, the real life budgetary and time limits rather than mathematical conditions often become 
de facto termination criteria. This favors the NAND approaches since they produce a succession of 
improving designs with meaningful analysis results available for each. In contrast, the SAND approaches 
produce a usable result only at the very end of their execution. 


VIII. Concluding Remarks 

The paper identified six fundamental approaches, named by three-part names referring to decomposition 
into levels and treatment of the variables: 1) Single-level vs multilevel optimization (first name Single 
vs Multi); 2) System-level simultaneous analysis and design vs analysis nested in optimization (middle 
name SAND vs AND); 3) Discipline-level simultaneous analysis and design vs analysis nested in 
optimization (last name SAND vs NAND). A compact notation was introduced for these approaches to 
define concisely the multitude of variations that may be developed by mixing, sequencing, and composing 
the approaches. Moreover, each approach may be implemented in a number of options whose defining 
features are techniques for the gradient computation and serial or parallel computing. The approaches 
which have previously been identified in the literature as limited to hierarchic systems have been shown 
to be applicable to nonhierarchic systems. 

The elapsed time (ET) was estimated for different implementation options and test cases. Certain 
conclusions become apparent from these limited results. First, no single approach is fastest for all test 
cases and implementation options. Conversely, no single approach can be identified as being always the 
slowest. Generally, the single-level optimization approaches are fastest on analysis-intensive problems 
while: the multilevel optimization approaches are fastest on design-intensive problems. This suggests 
mixing approaches by employing disciplinary optimizers only in disciplines with large numbers of design 
variables. Second, the choice of approach affects the ET as much or more than the choice of 
implementation (SI vs. PI, FD vs. DAS). Furthermore, the amount of savings in ET gained by going from 
SI to PI or from FD to DAS depends on the approach. In some cases the savings may be virtually 
nonexistent while in others they may be dramatic. 

These observations point to the recommendation that for a given real-world multidisciplinary problem 
that is expected to consume considerable resources, one should carefully consider the choice of approach. 
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It would be wise to estimate the computational effort for alternative approaches in much the same way 
as was illustrated for the examples given in this paper, and include memory requirements specific to the 
problem at hand. The managerial benefits of multilevel optimization approaches should also be 
considered. The corollary to the above is that if general-purpose system development is attempted to 
support multidisciplinary optimization in engineering design, that system should be flexible enough to 
offer the users the choice among approaches. 

Since the proper selection of approach is vital to the efficient solution of MDO problems, research is 
needed in the following areas for a better support of this selection process: 

1) Research into MDO fundamental approaches should continue to determine whether any exist that were 
not identified herein. 

2) The fundamental approaches should be demonstrated and compared on the same nonhierarchic multi- 
disciplinary example. 

3) The optimality equivalence of the fundamental approaches should be verified with mathematical rigor. 

4) Formulas and parameter values used to estimate computational effort should be refined and tested on 
numerical results. They should also include the memory requirements. 

5) Multilevel optimization approaches should be extended to allow informal and discrete methods for 
increased disciplinary flexibility. 

6) The convergence properties and computational effort of the SAND approaches should be investigated 
for various engineering disciplines. 
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Appendix: Elapsed Time (ET) 
Estimation Formulas 


NYj = ^j. l NT )i = number of coupling variables input to discipline i 
CAj = MA • NS 3 /3 
CEj = NS, 2 


Single-SAND-SAND 

CO = MO 1.5 • (NX+^iCNXj+NSj+NYj)) 3 

SI/FD: ET = MO • ^(CE, ' (l+NX+NX,+NS i +NY i ))+CO 

SI/DAS: ET=MO • ^(CE, ■ (l+NX+NXj+NY^+CO 

PI/FD: ET = MO • max,(CE,)+CO 

PI/DAS: ET = MO • max^CEO+CO 


Single-SAND-NAND 

CO = MO 1.5 • (NX+2 i (NX j +NY j )) 3 

SI/FD: ET = MO • ^(CA* * ( 1 +NX+NX i +NY,))+CO 

Sl/DAS: ET = MO • 2i(CA,+CE, • (NX+NX^NY^+CO 

PI/FD: ET = MO • max i (CA j )+CO 

Pl/DAS: ET = MO • max i (CA,+CE i )+CO 


Single-NAND-NAND 

CO = MO • 1.5 • (NX+JiNXi) 3 

CGSE = (SjNY^/S-KNX+JiNXj) • (2, NY,) 2 

SI/FD: ET = MO • (S,(MC • CA,+CA, • (NX+NX i +NY i ))+CGSE)+CO 

SI/DAS: ET = MO • (^(MC • CA 1 +CE i • (NX+NX i +NY i ))+CGSE)+CO 

PI/FD: ET = MO • raax,(MC • CA^+CO 

PI/DAS: ET = MO • (max^MC • CA i )+max i (CE i )+CGSE)+CO 
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Multi-SAND-SAND 


CO = MO • 1.5 • NX+SiNY;) 3 
CO, = MO • 1.5 • (NX.+NSi) 3 


SI/FD: ET = MO • 2i(MO • CE; • (l+NXj+NS^+COj+CE, • (NX+NY^+CO 

SI/DAS: ET =MO • 2, (MO • CE, • (l+NXJ+COj+CE, • (NX+NY,))+CO 

PI/FD: ET = MO ■ max,(MO • CE,+COj)+CO 

Pl/DAS: ET = MO ■ max,(MO • CE i +CO,+CE i )+CO 


Multi-SAND-NAND 

CO = MO ■ 1.5 • (NX+JiNYj) 3 
COj = MO • 1.5 • (NXj) 3 

SI/FD: ET = MO • 2i(M0 • CA, • (l+NX^+CO.+CA, • (NX+NY^+CO 

SI/DAS: ET = MO • 2, (MO • (CA.+CE, • NX,)+CO,+CE, • (NX+NY s ))+CO 

PI/FD: ET = MO • max, (MO • CAj+CO ; )+CO 

PI/DAS: ET = MO * max, (MO • (CA,+CE i )+CO i +CE i )+CO 


Multi-NAND-NAND 

CO = MO 1.5 • (NX+2i2jl) 3 

CO s = MO • 1.5 • (NXj) 3 

CGSE = (2 i NY i ) 3 /3+(NX+2 i NXj) • (2, NY,) 2 

SI/FD: ET = MO - (2i(MC • CAj+CAj • (NX+NX,+NY,)+CGSE+ I;(MO • CAj - (1+NX,) 

+COj+CAj • NX))+CO 

SI/DAS: ET = MO - (2,<MC • CAj+CE ,• (NX+NXj+NYj))+CGSE+ 2i(MO - (CA.+CE, • NXj) 

+COj+CEj • NX))+CO 

PI/FD: ET = MO - (max,(MC • CA,)+max,(MO • CAj+COj))+CO 

PI/DAS: ET = MO • (maXj(MC • CA,)4 ma-,(CE,)+CGSE+max,(MO - (CAj+CEj)+COj+CEj))+CO 
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Table 1. Aircraft Wing Multidisciplinary Optimization Problem: 
Examples of Variables and Design Functions 


Symbols 

Examples 

s i 

fuel required to fly a given mission 

S2 

flow velocities 

Sj 

nodal displacements 

yn 

angle of attack for various flight regimes 

y» 

fuel volume to be contained in the wing 

y 2 i 

aerodynamic drag for given angle of attack and Mach number 

yi3 

aerodynamic loads 

y 3 i 

structural weight 

y 32 

displacements that alter aerodynamic shape 

X 

wing planform geometry; airfoil depth-to-chord ratio 

Xi 

flight profile: Mach numbers vs. attitude data 

X 2 

airfoil geometry outside of structural box 

X 3 

cross-sectional dimensions of the wing structural box 

gl 

limits on the take-off and landing run lengths; required go-around climb rate; 
required range for a given payload 

g 2 

aerodynamic pressure distribution limits; angle of attack limit; 

g 3 

stress and displacement limits; required flutter velocity 

f, 

minimum take-off gross weight 

fa 

minimum drag 

f 3 

minimum structural weight 
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Table 2. Implementation Options 


Option 

SI 

PI 

FD 

OSA 

GSE 

DAS 

Designatio 

n 

1 

o 


o 

o 

o 


SI/FD 

2 

o 



o 

o 

o 

SI/DAS 

3 


o 

o 




PI/RD 

4 


o 


O 

o 

0 

PI/DAS 
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Fig. 1. A Three-Discipline Coupled System. 



Fig. 2. The Single-SAND-SAND Approach. 
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Fig. 3. The Single-SAND-NAND Approach. 
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Fig. 4. The Single-NAND-NAND Approach. 
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Fig. 5. The Multi-SAND-SAND Approach. 



Fig. 6. The Multi-SAND-NAND Approach. 
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Fig. 7. The Multi-NAND-NAND Approach. 



Fig. 8. Two-Discipline Case. 
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Fig. 9. Twenty-Discipline Case, Analysis-Intensive. 



Fig. 10. Twenty-Discipline Case, Design-Intensive. 
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